In the field of natural language processing, the prevalent approach involves fine-tuning pretrained language models (PLMs) using local samples.

Optimizing proper loss functions is popularly believed to yield predictors with good calibration properties; the intuition being that for such losses, the global optimum is to predict the ground-truth probabilities, which is indeed calibrated.

The training of language models is typically not an end-to-end process.

Generative Language Models (GLMs) have shown impressive performance in tasks such as text generation, understanding, and reasoning. However, the large model size poses challenges for practical deployment.

Neural Information Processing Systems http://nips.cc/